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1.  Introduction 


The  image  acquisition  model,  shown  in  figure  la,  demonstrates  the  measurement  model  for  the 
measured  signal  or  image  r(x,  y )  via  the  output  of  a  Linear  Shift  Invariant  (LSI)  system  whose 
impulse  response  is  the  sensor’s  point  spread  function  (PSF)  g(x,  y);  the  latter  is  also  known  as 
the  sensor  blurring  function.  The  signal  n(x,  y)  is  the  additive  noise.  The  observed  signal  or 
image  r(x,  y)  is  the  sum  of  the  noise  and  the  convolution  of  the  sensor  blurring  function  g(x,  y) 
with  the  “true”  signal  or  image  /(x,  v): 

r(x,  y)  =  f(x,  y)  ®  g(x,  y)  +  n(x,  y).  (1) 

The  convolution  relation  in  the  spatial  frequency  domain  ( kx ,  kv)  is  written  as  follows: 

R(kx,  ky)  =  F(kx ,  ky)  ■  G(kx,  ky)  +  N(kx,  ky )  (2) 

where  R(kx,  ky),  F(kx,  ky),  G(kx,  ky),  and  N(kx,  ky )  are  the  Fourier  transforms  of  r(x,  y),j[x,  y), 
g(x,  v),  and  n(x,  y),  respectively.  The  blurring  function  G(kx,  ky),  in  most  practical  imaging 
systems,  is  a  radially  symmetric  function.  In  that  case,  if  the  radial  spatial  frequency  domain  is 
defined  to  be 

P  =  yl^x  +K 5  (3) 

the  blurring  function  can  be  identified  as  G(p).  The  blurring  function  G( p)  is  typically  a  low- 
pass  filter  (e.g.,  Gaussian).  The  noise  n(x,  y)  is  a  white  process,  i.e.,  the  spectrum  |iV(/?)|2  has  a 
wide-bandwidth  (flat)  support  band  as  shown  in  figure  lb. 


The  deblurring  process  is  to  recover  fix,  y)  from  the  measured  signal  r(x,  y)  by  applying  the 
deblurring  filter  h(x,  y)  (figure  la).  This  process  is  also  called  deconvolution  or  restoration 
(1-3).  The  straightforward  deblurring  filter  is  the  inverse  of  the  blurring  function. 
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(4) 


When  we  apply  this  kind  of  deblurring  fdter,  we  have, 


Rip) 

G(p) 


=  F{p)  + 


Nip) 

G{p)' 


The  second  term  usually  blows  up  (becomes  unstable)  at  the  high  frequencies  where  the  signal 
level  \F{p)\  is  below  the  noise  level  |iV(/?)| . 

Several  recently  reviewed  journal  articles  ( 4-7)  have  summarized  the  research  that  attempts  to 
solve  this  problem.  Demoment  quotes,  “Everything  was  summed  up  in  a  single  word: 
regularization”  (4).  Deblurring  is  an  ill-posed  inverse  problem  due  to  the  above-mentioned 
instability.  For  ill-posed  (or  ill-conditioned)  problems,  obtaining  the  true  solution  from 
imperfect  data  is  impossible.  Regularization  is  a  general  principle  to  obtain  an  acceptable 
solution  by  incorporating  some  extra  or  a  prior  information  of  the  imaging  system. 

In  summary,  regularization  is  a  method  that  a  prior  knowledge  of  the  imaging  systems  (e.g.,  PSF 
of  the  system)  or  assumption  of  the  true  image  [fix,  y)]  is  utilized  to  form  a  risk  function.  The 
risk  function  is  then  solved  by  different  implementations  of  the  non-linear  optimization 
problems,  such  as  the  gradient-based  method,  expectation-maximization  (EM)  technique, 
prediction  error-based  technique,  and  least  square  methods.  Recent  research  (8,  9)  differs  in  the 
assumptions  about  the  true  image  and  PSF,  and  how  these  assumptions  are  imposed  in  forming 
different  risk  functions.  An  example  is  that  searching  the  solution  o [fix,  y)  value  which 
maximizes  the  a  posterior  (MAP)  distribution 

fix,  y)  =  arg  max  p[f(x,  y)\r{x,  y)].  (5) 


Usually,  the  risk  function  depends  on  the  actual  object  which  is  unknown.  An  appropriate 
solution  has  to  be  chosen  through  proper  initialization  of  the  algorithm.  Because  of  the  nature  of 
non-linear  optimization  problems,  there  is  a  chance  that  the  methods  may  converge  to  local 
minima.  In  addition,  the  convergence  speed  may  be  slow. 


Additionally,  recent  research  has  been  using  the  various  implementations  of  the  Wiener  filter 
approach.  The  Wiener  filter  is  defined  as 


H 


■ip)  = 


GipY 


pipf  +  Sn  ip)  /  Sf  ip) 


(6) 


where  S/p)  and  Sn(p)  are  the  signal  and  noise  power  spectral  densities,  respectively. 
Theoretically,  the  Wiener  filter  is  the  optimal  linear  filter  (in  Root  Mean  Square  Error  (RMSE) 
sense).  The  problem  with  the  Wiener  filter  is  that  its  parameters  (degradation  function  and  signal 
to  noise  ratio  (SNR))  are  not  always  known  and  in  such  case,  it  suffers  from  limitations  such  as 
those  to  be  discussed  next. 
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When  the  noise  power  is  weak  ( S„(p)/Sj(p )  «  1),  the  Wiener  filter  becomes  \/G(p),  that  is, 
pseudo  inverse  of  G(p),  which  has  the  instability  problems  previously  mentioned.  Theoretically, 
if  G(p)  and  the  SNR  are  known  precisely,  then  the  instability  is  negligible  (in  RMSE  sense)  since 
the  noise  is  negligible.  However,  in  practice  the  SNR  and  G(p)  are  not  known  precisely  and  the 
estimated  inverse  filter  1  /G(p)  may  be  instable  because  of  small  errors  in  the  estimated  filter 
parameters.  On  the  other  hand,  when  the  noise  is  strong  ( S„(p)/Sj(p )  »  1),  the  Wiener  filter 
becomes  G(p)*/[Sn(p)/Sf(p)\,  that  is  proportional  to  the  SNR,  which  filters  noise,  but  not 
effective  for  deblurring. 


The  parametric  Wiener  filter  is  implemented  using  the  parameter  yin  the  following  definition: 


H„ 


Xp) 


G{Py _ 

G(pf  +  Y  Sn(p)  /  S  f(p)  ' 


(7) 


The  effect  of  yon  the  filter  tends  to  emphasize  (y>  1)  or  de-emphasize  (y<  1)  the  noise  and 
signal  statistics,  respectively.  The  value  of  ymust  be  chosen  carefully  for  reliable  deblurring. 

The  choice  of  yis  achieved  through  an  iterative  search  (5).  The  uniqueness  and  convergence  are 
uncertain  and  the  restoration  is  sensitive  to  the  initial  image  estimate.  Combining  an  inverse 
filter  with  a  weighting  filter  in  a  geometric  mean  filter  (10)  and  the  digital  restoration  in  the 
wavelet  domain  approach  (11)  are  also  reported  in  the  literature. 

In  this  report,  we  design  a  practical  deblurring  filter  that  is  less  sensitive  to  the  noise,  while 
removing  the  blurring  by  a  desired  amount.  We  call  it  the  P-deblurring  filter  because  we  utilize  a 
special  window,  called  a  “power”  window,  to  construct  the  deblurring  filter.  The  deblurring 
process  applies  the  P-deburring  filter  to  the  input  image  in  the  spatial  frequency  domain:  the 
method  does  not  require  an  iterative  approach.  Section  2  describes  the  structure  of  the  P- 
deblurring  filter.  The  design  method,  presented  in  section  3,  utilizes  the  filter  properties  and 
estimates  the  energy  of  the  signal  and  noise  of  the  input  image.  The  P-deblurring  filter’s 
performance  is  demonstrated  by  a  human  perception  experiment  which  is  described  in  section  4. 
The  result  analysis  and  conclusion  are  summarized  in  sections  5  and  6. 


2.  P-Deblurring  Filter 


The  blurring  function  is  usually  a  Gaussian-like  function  having  a  bell  shape.  The  inverse 
function’s  magnitude  increases  at  high  frequencies  where  the  noise  can  be  magnified 
enonnously.  In  order  to  suppress  the  noise  effect,  we  apply  a  special  window  called  a  power 
window,  to  the  inverse  blurring  function.  The  power  window  (12)  is  a  Fourier-based  smooth 
window  that  preserves  most  of  the  spatial  frequency  components  in  the  passing  band  and 
attenuates  quickly  at  the  transition-band.  The  power  window  is  differentiable  at  the  transition 
point,  which  gives  a  desired  smooth  property  and  limits  the  ripple  effect.  A  power  window,  and 
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a  deblurring  filter  when  applying  the  power  window  to  a  Gaussian  blurring  filter,  are  shown  in 
figure  2.  Our  deblurring  filter  is  called  the  P-deblurring  filter  because  it  was  derived  from 
utilizing  the  power  window. 


Figure  2.  P-deblurring  filter  properties. 

For  a  given  blurring  function  G(p),  the  P-deblurring  filter  P(p)  is  written  as: 

P(p)  =  P--W(p)  (8) 

G(p) 

where  W(p)  is  the  power  window.  The  power  window  is  defined  as  follows: 

W  ( p )  =  exp(-a  p" )  (9) 

where  the  parameters  {a,  n)  are  obtained  based  on  certain  specifications.  Since  the  parameter  n 
is  the  power  of  the  smooth  window,  this  window  is  called  power  window.  When  we  assume  the 
Gaussian  blurring  function,  the  P-deblurring  filter  is  defined  as 

jp(p)  =  ^M  =  exp(-«p"+^)  (10) 

G{p )  2  a2 

where  cr  is  the  standard  deviation  of  the  Gaussian  function. 
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3.  P-Deblurring  Filter  Design 


In  this  section,  we  describe  the  properties  of  the  P-deblurring  filter  derived  from  the  power 
window.  Our  design  methods,  of  the  P-deblurring  filter  based  on  these  properties  and  user’s 
specifications,  are  then  proposed. 

3.1  Properties  of  the  P-Deblurring  Filter 

3.1.1  The  Peak  Point 


The  P-deblurring  filter  has  a  maximum  magnitude  at  peak  point  as  shown  in  figure  2.  This  peak 
point  is  solved  by  setting  the  derivative  of  the  P-deblurring  filter  zero.  That  is, 


oPjp) 

dp  Pp 


0  =>  Pp 


1 

(ana2)2'1. 


(ID 


The  maximum  magnitude  of  the  P-deblurring  filter  is  an  important  parameter  for  the  filter 
design.  Here,  the  peak  point  pp  is  a  function  of  three  parameters  (a,  n,  a).  The  filter  design 
criteria  are  derived  from  specifying  these  parameters. 


3.1.2  The  Noise  Separation  Frequency  Point 


Usually,  the  signal  is  stronger  than  noise  in  the  low  frequencies  and  weaker  than  noise  in  the 
higher  frequencies.  The  noise  is  assumed  to  be  a  white  process  cross  a  wide  range  of  the 
frequency.  At  one  point,  called  the  noise  separation  frequency  point  pn,  the  signal  and  noise  are 
equal  as  shown  in  figure  lb.  The  noise  is  stronger  than  the  signal  for  frequencies  greater  than  pn. 


One  criteria  of  the  filter  design  is  to  allow  the  magnitude  of  the  P-deblurring  filter  to  be  1  at  the 
noise  separation  frequency  point  pn  as  shown  in  figure  2.  In  this  way,  the  magnitude  of  the 
P-deblurring  filter  is  less  than  1  for  frequencies  greater  than  the  noise  separation  frequency  point 
pn  where  the  noise  is  stronger  than  the  signal.  Meanwhile,  the  magnitude  of  the  P-deblurring 
filter  can  be  large  below  the  noise  separation  frequency  point  pn  where  the  signal  is  strong.  This 
kind  of  deblurring  filter  provides  the  necessary  deblurring.  Therefore,  the  P-deblurring  filter 
satisfies  the  following  condition: 

>1,  if  p  <  pn ; 

P{p)  =  \  =  1,  ifp  =  pn\  (12) 

<1,  if  p>  pn. 


If  the  SNR  at  the  noise  separation  frequency  point  pn  is  estimated  (see  subsection  3.3),  we  can 
also  solve  three  parameters  (a,  n,  a)  from  (10)  using  pn  and  P(p„)  =1. 
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3.1.3  The  Cutoff  Frequency  Point 


The  cutoff  frequency  point  of  the  power  window,  po,  is  where  the  magnitude  of  the  power 
window  attenuates  significantly.  This  point  has  an  important  impact  on  the  filter  design  since  it 
controls  when  the  inverse  blurring  function  drops  significantly.  The  cutoff  frequency  can  also 
detennine  the  shape  of  the  power  window  by  solving  the  two  parameters  ( a ;  n)  (12). 

3.2  Direct  Design 

In  this  section,  we  describe  a  direct  design  method  of  the  deblurring  filter  which  can  benefit  an 
imaging  system  that  the  sensor  information  and  the  noise  characteristic  are  known.  The  direct 
design  method  is  simple  and  easy  to  implement.  The  performance  of  this  deblurring  filter  is 
predictable. 

Assume  the  blurring  function  is  a  Gaussian  function.  We  can  specify  the  magnitude  of  the 
inverse  Gaussian,  mg,  at  one  frequency  point,  pg.  This  frequency  point  pg  is  calculated  in  the 
following: 

*  .  =  mg  =>  P,  =  trpln (mg)  (13) 

G(p,) 

where  a  is  the  standard  deviation  of  the  Gaussian  function.  The  standard  deviation  of 
Gaussian  cr  is  either  known  or  estimated  by  applying  a  curve  fitting  procedure  to  the  ensemble 
magnitude  spectrum  (see  subsection  3.3.1).  If  we  choose  the  cutoff  frequency  point  p0  as  pg,  the 
power  window  is  constructed.  The  P-deblurring  filter  is  finally  determined  from  (9). 

An  example  of  applying  the  P-deblurring  filter  to  one  Forward-Looking  Infrared  (FLIR)  tank 
image  is  shown  in  figure  3.  Note  the  numbers  of  pixels  inx  and  y  domains  are  different. 
However,  the  spatial  sampling  spaces,  Ax  and  Ay,  are  the  same.  From  the  discrete  Fourier 
transform  (DFT)  equation  (13), 


Nx 

•A, 

=  In 

(14) 

Ny 

=  2  n 

(15) 

where  Nx  and  Ny  are  the  numbers  of  pixels  in  x  and  y  domains,  respectively;  Ak  and  Ak  are  the 
sampling  spaces  in  spatial  frequency  domains,  respectively.  Thus,  Ak  and  At  are  not  equal  to 
each  other.  However,  the  support  bands  in  the  spatial  frequency  domains, 


are  equal  to  each  other.  For  this  reason,  the  spectral  distributions  shown  in  3c  and  3d  are  square 
shaped  and  not  rectangular  shaped  as  in  the  spatial  domain  image. 


Figure  3.  Example  of  the  direct  design  for  a  FLIR  tank  image. 

Note:  (a)  Original  (blurred),  and  (b)  P-deblurred  images  are  illustrated 
in  spatial  domain,  (c)  Gaussian  filter,  and  (d)  P-deblurring 
filters  are  illustrated  in  spatial  frequency  domain. 

We  assume  that  this  image  is  already  blurred  by  a  Gaussian  function.  By  selecting 

a  =  0.25  *  pmax,  mg  =  10  (18) 


where  pmax  is  the  maximum  frequency  of  the  Fourier  transform  of  the  input  image.  The  original 
and  P-deblurred  tank  images  are  shown  figure  3a  and  3b.  The  Gaussian  blurring  function  and 
the  resultant  P-deblurring  filter  are  shown  in  figure  3  c  and  d.  The  deblurred  tank  clearly  shows 
the  deblurring  improvement,  especially  around  the  tire  area.  Figure  4  shows  the  one-dimensional 
(1-D)  cross  section  of  each  image  in  figure  3.  The  1-D  cross  section  of  the  tank  image  is  a 
horizontal  crossing  line  at  the  center  of  the  tires.  The  deblurring  effects  show  equally. 
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3.3  Adaptive  Design 

In  this  section,  we  describe  a  deblurring  filter  that  is  adaptively  designed  based  on  the  estimated 
energy  of  the  signal  and  noise  of  the  image,  which  can  benefit  an  imaging  system  with  noise. 

The  adaptive  design  estimates  the  noise  energy  and  the  noise  separation  frequency  point  where 
the  energy  of  signal  and  noise  are  equal.  The  noise  energy  is  stronger  than  the  signal  energy 
beyond  that  noise  separation  frequency  point.  The  filter  design  criteria  are  defined  to  obtain  a 
deblurring  filter  such  that  the  filter  magnitude  is  less  than  1  at  the  frequencies  above  the  noise 
separation  frequency  point;  the  filter  magnitude  is  larger  than  1  at  the  frequencies  below  the 
noise  separation  frequency  point  and  the  signal  is  strong  as  shown  in  (12).  Therefore,  the 
adaptively  designed  P-deblurring  filter  is  able  to  deblur  the  image  by  a  desired  amount  based  on 
the  known  or  estimated  standard  deviation  of  the  Gaussian  blurring  while  suppressing  the  noise 
in  the  output  image. 

3.3.1  Estimating  Noise  Energy  and  Noise  Separation  Frequency  Point 

In  this  section,  we  outline  a  method  to  estimate  the  noise  energy  level  in  the  spectral  domain  of 
the  acquired  image.  Let /(x,  y)  be  the  continuous  input  image.  We  define  its  1-D  marginal 
Fourier  transform  via 

Fx(kxt  y )  =  3,v)[/(x,  y)].  (19) 
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Now  consider  the  image  that  is  the  discrete  version  of  fix,  y ),  that  is,  fi],i  =  \,---,N,j  =  \,---,M. 
The  1-D  DFT  of  this  two-dimensional  (2-D)  array  are  discrete  samples  of  Fx(kx,  y )  : 

F.w , i  =  1,  •  •  •  N,  j  =  1, •  •  -M.  The  ensemble  magnitude  spectrum,  S  =  , •  •  •  sN ] ,  is  computed  via 


i  =  N. 


(20) 


An  example  of  the  ensemble  magnitude  spectrum  from  one  input  FLIR  image  is  shown  in  the 
upper  left  corner  of  figure  5a. 
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Figure  5.  Ensemble  magnitude  spectrum  of  a  FLIR  image. 


Usually,  the  signal  is  stronger  than  the  noise  at  the  low  frequencies  and  weaker  than  the  noise  at 
the  high  frequencies.  The  same  figure  shows  the  ensemble  magnitude  spectrum  remained 
unchanged  after  a  high  frequency  point.  The  unchanged  spectrum  is  mainly  a  noise  spectrum 
since  the  noise  dominates  the  signal  at  the  very  high  frequencies.  Furthennore,  the  noise  is 
commonly  a  white  process  cross  a  wide  range  of  frequencies.  The  noise  separation  frequency 
point  is  the  frequency  where  the  ensemble  spectrum  becomes  unchanged  (levels  off). 
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In  order  to  accurately  estimate  the  noise  separation  frequency  point,  we  first  smooth  the 
ensemble  spectrum  by  convolving  it  with  an  averaging  filter;  denoted  by  SM : 

SM  =[smx,---  ,smN],  (21) 

The  averaging  filter  is  chosen  to  be 

[1  1  1]  /  3  (22) 

as  an  example.  After  this  step,  the  gradient  of  the  smoothed  spectrum  is  computed  as  follows: 

Gi=smM-smi_v  (23) 

Examples  of  the  smoothed  spectrum  (5b)  and  its  gradient  curve  for  the  positive  frequencies  (5c) 
are  shown  in  figure  5 .  The  gradient  curve  is  searched  from  zero  frequency  to  the  positive 
frequencies.  The  search  stops  at  a  frequency  point  when  the  absolute  value  of  the  gradient  is  less 
than  a  threshold;  we  refer  to  this  as  the  index  frequency  point.  Note  that  the  selection  of  this 
threshold  is  suggested  to  have  the  index  frequency  point  to  be  at  the  beginning  point  when  the 
ensemble  spectrum  levels  off.  The  smooth  filtering  of  the  ensemble  spectrum  makes  the 
threshold  less  sensitive  to  the  noise. 

The  value  of  the  ensemble  magnitude  spectrum  at  the  index  frequency  point  represents  the  noise 
energy;  the  value  of  the  ensemble  magnitude  spectrum  near  the  zero  frequency  point  represents 
the  signal  energy.  The  SNR  is  computed  as  the  ratio  of  the  signal  energy  to  the  noise  energy. 
Note  the  value  of  the  ensemble  magnitude  spectrum  at  the  zero  frequency  represents  the  direct 
current  (DC)  value  of  the  image  energy;  therefore,  the  value  is  usually  a  very  big  number.  The 
calculation  of  SNR  should  avoid  using  this  value  and  instead  use  the  value  nearer  the  zero 
frequency  point.  Using  the  value  of  the  ensemble  magnitude  spectrum,  that  is  1-2  pixels  from 
zero  frequency,  is  a  good  practice. 

The  spectrum  of  an  ideal  (point)  target  is  flat;  however,  the  acquired  spectrum  of  a  target  is 
decayed  due  to  the  blurring  function.  At  the  index  frequency  point,  the  acquired  signal  value  is 
Ysnr  if  the  ideal  signal  is  1 .  The  noise  separation  frequency  point  pn  is  calculated  as  follows: 

pn  =  ap  In  (SNR),  (24) 

where  a  is  the  standard  deviation  of  the  Gaussian  function. 
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3.3.2  Filter  Design 


The  overall  goal  of  the  deblurring  filter  is  to  provide  the  satisfied  output  image,  i.e.,  the  proper 
deblurring  and  the  proper  noise  control.  This  goal  can  be  achieved  by  adaptively  determining  the 
proper  parameters  of  the  deblurring  filter  based  on  estimated  noise  energy  of  the  input  image. 

As  earlier  stated,  to  construct  the  P-deblurring  filter  we  need  to  determine  three  parameters 
(a,  /?,  a).  From  the  discussion  above,  the  standard  deviation  of  Gaussian  a  is  either  known  or 
estimated  by  applying  a  curve  fitting  procedure  to  the  ensemble  magnitude  spectrum.  By 
specifying  two  distinct  values  of  P(p)  as  shown  in  (9),  we  can  solve  for  the  other  two  parameters 
(a,  n).  For  example,  one  can  select: 


p  U-*=  1. 

P  I p2  =o.spn  —  W2- 


(25) 


This  means  that  the  magnitude  of  the  P-deblurring  filter  is  set  to  1  at  the  noise  separation 
frequency  point,  and  the  magnitude  is  set  to  W2  ( W2  >  1)  at  80%  of  the  noise  separation 
frequency  point. 

However,  the  selection  of  W2  is  not  completely  arbitrary.  Because  the  power  window  is  less  than 
1 ,  the  magnitude  of  the  P-deblurring  filter  is  less  than  the  inverse  Gaussian  function,  that  is, 


W2  <  exp(^V). 
la 


(26) 


Therefore,  W2  is  chosen  according  to  this  condition.  There  are  other  procedures  to  detennine  the 
other  two  parameters  (a,  n ).  Caution  must  be  used  when  detennining  these  parameters,  since  the 
conditions  of  P-deblurring  filter  need  to  be  met.  An  example  of  applying  the  adaptive  design 
P-deblurring  filter  to  a  FLIR  image  is  shown  in  section  4. 


4.  Perception  Experiment 


In  order  to  assess  the  performance  of  our  proposed  adaptive  deblurring  filter  method,  a 
perception  experiment  was  conducted  to  demonstrate  the  P-deblurring  filter  effect  on  the  target 
identification  task  of  humans.  The  image  set  consisted  of  12  standard  targets  taken  from  a  field 
test  by  Night  Vision  and  Electronic  Sensors  Directorate  (NVESD)  using  a  longwave  infrared 
sensor.  The  targets  in  the  set  are  well-characterized  in  terms  of  predicting  range  performance 
based  upon  sensor  characteristics.  The  side  views  of  these  12  targets  (at  the  aspect  angle  90 J  or 
270°)  are  depicted  in  figure  6.  Each  target,  posing  at  12  aspect  angles,  makes  the  total  image  set 
contain  144  images  of  12  different  targets. 
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Target_05  Target_06  Target_07  Targeted  8 


Figure  6.  Thermal  image  examples  for  12-target  sets. 

Experimental  cell  organization  was  based  on  three  factors.  First,  images  were  subjected  to  six 
levels  of  blur  to  simulate  range  effects  on  identification  task  difficulty.  Second,  images  were 
subjected  to  three  different  noise  treatments  in  order  to  test  the  effectiveness  of  the  deblurring 
algorithm  on  noisy  images.  Third,  each  of  the  cells  was  processed  using  the  deblurring 
algorithm  to  quantify  differences  between  blurred  and  deblurred  image  cases  as  a  function  of 
blur.  An  illustration  of  the  experimental  matrix  layout  and  naming  convention  is  shown  in 
table  1. 


Table  1 .  Experiment  matrix  layout  and  naming  convention. 
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The  total  144  images  in  the  first  row  are  arranged  into  six  blur  cells  in  the  following  manner. 
The  aspect  angles  were  rotated  through  the  blur  cells  so  that  each  cell  had  the  same  number  of 
images  at  a  particular  aspect  angle.  Within  each  blur  cell,  there  are  24  target  images  that  have 
various  aspect  angles  of  12  different  targets.  It  should  be  noted  that  the  target  images  within  a 
particular  blur  in  the  first  row  are  the  same  target  images  for  that  particular  blur  in  the  noise 
adding  and  deblurring  cells  in  the  same  column. 

Figure  7  shows  Gaussian  fdters  with  six  values  of  standard  deviations  that  were  applied  to 
imagery  to  generate  blurred  images.  These  six  standard  deviations  of  Gaussian  filters  were 
chosen  to  vary  the  observer  performance,  from  a  very  high  probability  of  a  correct  answer  to  a 
very  low  probability  of  a  correct  answer,  with  equally  decreasing  increment. 


Figure  7.  Gaussian  function  in  frequency  domain. 

Three  noise  treatments  followed.  First,  there  was  no  additional  noise  added  to  the  blurred 
images.  This  case  is  considered  a  low  noise  situation,  and  noted  as  NoiseO.  For  the  other  two 
noise  treatments,  two  levels  of  white  noise  were  added  to  the  blurred  images.  A  white  noise  with 
a  standard  deviation  of  100  millikelvin  (mK)  is  added  to  a  typical  infrared  image,  which  has  the 
signal  to  noise  ratio  of  20  K  and  considered  to  be  the  upper  bound  of  noise.  Based  on  this,  the 
second  noise  case,  noted  as  Noise  1,  is  chosen  to  be  a  white  noise  with  a  standard  deviation  of  50 
mK,  which  is  a  typical  noise  situation.  The  third  noise  case,  noted  as  Noise2,  is  the  white  noise 
with  a  standard  deviation  of  100  mK,  which  is  an  upper  bound  noise  situation.  An  image  that 
was  blurred  with  blur  level  6  and  the  blurred  image  with  added  noise  are  shown  in  the  first  row 
of  figure  8.  The  P-deblurred  resultant  images  are  shown  in  the  second  row  of  figure  8.  The 
P-deblurred  images  demonstrate  deblurring  improvement  from  their  blurred  input  images, 
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especially,  when  the  noise  added  in  the  blurred  image  is  suppressed  in  the  deblurred  output 
image,  which  benefits  from  the  adaptive  design  method. 

Target_09 


Blur4  Blur4+Noise1  Blur4+Noise2 


P-deblur  P-deblur  P-deblur 


Figure  8.  Sample  examples  of  blurred,  blurred  with  noise,  and  P-deblurred  images. 


Ten  observers  were  trained  using  the  ROC-V  Thermal  Recognition  software  to  identify  the 
12  target  set  as  presented  in  the  experiment.  Each  observer  was  trained  to  a  96%  proficiency  in 
identification  of  pristine  images  at  various  ranges.  This  proficiency  requirement  prevents 
potentially  biased  results  stemming  from  subjects  learning  target  specific  characteristics  during 
the  testing  process. 

Experimental  trials  were  conducted  using  a  high  resolution,  medical  grade  display  in  order  to 
allow  for  accurate  representation  of  image  characteristics.  For  consistency,  the  displays  were  all 
calibrated  between  experimental  setups  using  a  Prichard  model  1980  spot  photometer  to  ensure 
(realistically)  identical  minimum,  mean,  and  peak  luminance.  Each  display  was  calibrated  to  a 
mean  level  of  5.81  foot-Lamberts.  Experimental  trials  were  conducted  in  a  darkened 
environment  and  observers  were  given  ample  time  to  adapt  to  ambient  lighting  conditions. 
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5.  Result  Analysis 


The  average  probabilities  of  identification  from  the  experiment,  P,  were  adjusted  using  the 
following  equation 


R 


id 


(27) 


to  remove  the  probability  that  an  observer  could  select  the  proper  response  simply  through 
guessing  1  out  12  targets.  Previous  experimental  analysis  indicates  that  the  inclusion  of  a  “sweat 
factor”nonnalizes  probabilities  to  90%  for  inclusion  of  a  10%  mistake  rate.  For  the  typical 
observer  this  normalization  provides  a  more  accurate  reflection  of  observer  performance  by 
accounting  for  mistakes  and  unfamiliarity  with  the  testing  environment. 

The  adjusted  probability  of  identification  PK|  as  a  function  of  blur  (range)  for  the  different  cases  is 
shown  in  figure  9.  There  was  a  significant  improvement  in  human  performance,  as  a  function  of 
range,  when  the  blurred  imagery  was  processed  using  the  P-deblurring  filter.  NoiseO  Blur 
images  are  considered  as  the  “baseline”  images.  For  the  blur  level  1-3  (near  range),  the  Pidof  the 
baseline  images  is  high  (60%  to  90%).  Flowever,  for  the  blur  level  4-6  (long  range),  the  P,d  of 
the  baseline  images  is  very  low  (10%  to  30%);  that  is  where  the  P-deblurring  filter’s 
improvement  takes  place  (38%  to  65%).  For  the  blur  level  4-6,  the  average  increase  of  P;d  is 
33%  by  using  deblurred  images  vs.  blurring  images  in  the  NoiseO  case  (figure  9a);  the  average 
increase  of  Py  is  15%  in  the  Noise  1  case  (figure  9b).  In  the  Noise2  case,  the  improvement  of  Py 
is  negligible  using  the  deblurred  images  (figure  9c).  In  summary,  the  Pid  is  increased  15%  to 
33%  for  the  case  of  typical  noise  to  low  noise  in  the  long  range  using  the  P-deblurring  filter 
method. 
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In  order  to  estimate  the  range  performance,  we  assume  the  range  is  linearly  increased  with  the 
increasing  blur  level  since  the  chosen  blurs  generate  equal  increment  blur  effect  on  the  images. 

In  figure  9a,  the  NoiseO  case,  blurring  imagery  had  a  50%  Pid  at  the  range  55%  unit  (blur  level 
3.3),  while  the  P-deblurred  imagery  reached  the  range  92%  unit  (blur  level  5.5)  for  the  50%  Pid. 
That  is  equivalent  to  67%  increase  in  range  performance.  Similarly,  in  the  Noise  1  case  shown  in 
figure  9b,  P-deblurred  imagery  yielded  the  range  67%  unit  (blur  level  4)  comparing  with  the 
range  58%  unit  (blur  level  3.5)  from  the  blurring  imagery  at  50%  P;d.  That  is  about  16%  increase 
in  range  performance.  In  the  Noise2  case,  at  50%  Pid,  the  range  is  at  63%  unit  (blur  level  3.8)  for 
P-deblurring  imagery  comparing  with  58%  unit  (blur  level  3.5)  from  blurring  imagery,  which  is  a 
9%  increase  in  range  perfonnance.  In  summary,  the  range  performance  is  increased  9%,  16%, 
and  57%  for  the  case  of  upper  bound  noise,  typical  noise,  and  low  noise  in  the  input  images, 
respectively,  using  the  P-deblurring  filter  method.  This  quantitative  analysis  of  the  range 
performance  is  estimated  approximately.  The  accurate  analysis  will  be  based  on  accurate  range 
calculation.  The  ideal  case  would  acquire  the  imagery  using  real  range  images,  which  is  our  plan 
for  future  research. 
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6.  Conclusions 


This  report  proposes  an  adaptive  deblurring  filtering  algorithm  for  noisy  images.  The  algorithm 
is  practical,  Fourier-based,  and  can  be  implemented  in  real-time.  The  improvement  using  this 
algorithm  for  noisy  blurring  images  is  demonstrated  in  a  human  perception  experiment  of  the 
target  identification  task.  The  potential  benefit  of  this  practical  deblurring  algorithm  is  that  it  can 
be  implemented  into  a  real  imaging  system  to  compensate  the  sensor  blurring  and  noise  effect  on 
its  output  images.  Another  potential  benefit  is  that  if  other  blur  sources  such  as  motion, 
vibrations,  or  atmosphere  can  be  modeled  as  a  Modulation  Transfer  Function  (MTF),  the 
proposed  adaptive  deblurring  filtering  algorithm  can  still  be  applied.  Since  the  power  window 
can  be  applied  to  a  known  inversed  MTF,  with  respect  to  the  estimated  blur  and  noise,  the  noise 
can  be  suppressed  in  the  deblurring  process  while  the  deblurring  can  be  performed  effectively. 
The  proposed  adaptive  deblurring  filtering  method  will  work  well  in  the  cases,  even  when  MTF 
is  not  monotonically  decreasing,  as  long  as  the  inverse  of  MTF  is  valid. 
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Acronyms  and  Abbreviations 


DC 

direct  current 

DFT 

discrete  Fourier  transform 

EM 

expectation-maximization 

FLIR 

Forward-Looking  Infrared 

K 

kelvin 

LSI 

Linear  Shift  Invariant 

MAP 

maximum  a  posterior 

mK 

millikelvin 

MTF 

Modulation  Transfer  Function 

NVESD 

Night  Vision  and  Electronic  Sensors  Directorate 

PSF 

point  spread  function 

RMSE 

root  mean  square  error 

SNR 

signal  to  noise  ratio 

1-D 

one-dimensional 

2-D 

two-dimensional 
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1  ADMNSTR 

(PDF  DEFNS  TECHL  INFO  CTR 
COPY)  ATTN  DTIC  OCP 

8725  JOHN  J  KINGMAN  RD  STE  0944 
FT  BELVOIR  VA  22060-6218 

2  DARPA 

ATTN  IXO  S  WELBY 
ATTN  J  RICKLIN 
3701  N  FAIRFAX  DR 
ARLINGTON  VA  22203-1714 

1  NGA 

ATTN  R  S  RAND 
12310  SURSISE  VALLEY  DR 
MAIL  STOP  DN  1 1 
RESTON  VA  20191-3449 

1  OFC  OF  THE  SECY  OF  DEFNS 

ATTN  ODDRE  (R&AT) 

THE  PENTAGON 
WASHINGTON  DC  20301-3080 

1  US  ARMY  RSRCH  DEV  AND  ENGRG 
CMND 

ARMAMENT  RSRCH  DEV  AND  ENGRG 
CTR 

ARMAMENT  ENGRG  AND  TECHNLGY 
CTR 

ATTN  AMSRD  AAR  AEF  T  J  MATTS 
BLDG  305 

APG  MD  21005-5001 

2  US  ARMY  TRADOC 

BATTLE  LAB  INTEGRATION  &  TECHL 

DIRCTRT 

ATTN  ATCD  B 

ATTN  ATCH  B 

10  WHISTLER  LANE 

FT  MONROE  VA  23651-5850 

1  US  MILITARY  ACDMY 

MATHEMATICAL  SCI  CTR  OF 
EXCELLENCE 
ATTN  MAJ  J  HARTKE 
PHOTONICS  CENTER 
WEST  POINT  NY  10996-1786 


1  PRODUCT  MANAGER  TIMS 

ATTN  SFAE  IEWS  NS  TIMS 
B  GRIFFIES 
BLDG  563 

FT  MONMOUTH  NJ  07703 

1  SMC/GPA 

2420  VELA  WAY  STE  1866 
EL  SEGUNDO  CA  90245-4659 

1  US  ARMY  ABERDEEN  TEST  CENTER 

ATTN  CSTE  DT  AT  WC  A  F  CARLEN 
400  COLLERAN  ROAD 
APG  MD  21005-5009 

1  US  ARMY  ABERDEEN  TEST  CENTER 

ATTN  CSTE  DTC  AT  TC  N 
D  L  JENNINGS 
400  COLLERAN  ROAD 
APG  MD  21005-5059 

1  US  ARMY  ERDC 

ATTN  CEERD  TR  S 

7701  TELEGRAPH  RD  BLDG  2592 

ALEXANDRIA  VA  22315 

1  US  ARMY  INFO  SYS  ENGRG  CMND 

ATTN  AMSEL  IE  TD  F  JENIA 
FT  HUACHUCA  AZ  85613-5300 

3  US  ARMY  MATERIEL  SYS  ANAL 

ACTVTY 

ATTN  AMSRD  AMS  SC  G  KISTNER 
ATTN  AMSRD  AMS  SC  J  MAZZ 
ATTN  AMSRD  AMS  SC  R  WHEELER 
392  HOPKINS  RD 
APG  MD  21005-5071 

1  US  ARMY  NATICK  RDEC 

ACTING  TECHL  DIR 
ATTN  SBCN  TP  P  BRANDLER 
KANSAS  STREET  BLDG  78 
NATICK  MA  01760-5056 
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1  US  ARMY  PM  NV/RSTA 

ATTN  SFAE  IEW&S  NV 
10221  BURBECKRD 
FT  BELVOIR  VA  22060-5806 

1  COMMANDER 

US  ARMY  RDECOM 

ATTN  AMSRD  AMR  W  C  MCCORKLE 

5400  FOWLER  RD 

REDSTONE  ARSENAL  AL  35898-5000 

1  US  ARMY  RDECOM  AMRDEC 
ATTN  ATCD  B 

FT  MONROE  VA  23651-5850 

2  US  ARMY  RDECOM  AMRDEC 
ATTN  AMSRD  AMR  SG  W  PITTMAN 
ATTN  AMSRD  AMR  SG  IP 

H  F  ANDERSON 
BLDG  5400 

REDSTONE  ARSENAL  AL  35809 

1  US  ARMY  RDECOM  AMRDEC 

ATTN  AMSRD  AMR  SG  IR  R  SIMS 
BLDG  5400 

REDSTONE  ARSENAL  AL  35898 

1  US  ARMY  RDECOM  AMRDEC 
ATTN  AMSRD  AMR  WS  PL 

W  DAVENPORT 
BLDG  7804 

REDSTONE  ARSENAL  AL  35898 

2  US  ARMY  RDECOM  ARDEC 
ATTN  AMSRD  AAR  AEP  S 

P  WILLSON 

ATTN  AMSRD  AAR  AER  S 
J  ROMANO 

RADIOGRAPHIC  LABORATORY  B  908 
PICATINNEY  ARSENAL  NJ  07806-5000 

4  US  ARMY  RDECOM  CERDEC  NVESD 

ATTN  AMSRD  CER  NV  MS 
R  DRIGGERS 

ATTN  AMSRD  CER  NV  OD 
J  RATCHES 

ATTN  AMSRD  CER  NV  ST  J  HILGER 
ATTN  AMSRD  CER  NV  ST 
P  PERCONTI 

10221  BURBECK  RD  STE  430 
FT  BELVOIR  VA  22060-5806 


1  US  ARMY  RDECOM  TARDEC 

ATTN  AMSRD  TAR  R  G  R  GERHART 
6501  E  ELEVEN  RD  MS  263 
WARREN  MI  48397-5000 

1  US  ARMY  RSRCH  LAB 

ATTN  AMSRD  ARL  Cl  OK  TP 
TECHL  LIB  T  LANDFRIED 
BLDG  4600 
APG  MD  21005-5066 

4  DIRECTOR 

US  ARMY  RSRCH  LAB 
ATTN  AMSRD  ARL  RO  EL  L  DAI 
ATTN  AMSRD  ARL  RO  M  D  ARNEY 
ATTN  AMSRD  ARL  RO  MI 
R  ZACHERY 

ATTN  AMSRD  ARL  RO  MM 
M-H  CHANG 
POBOX  12211 

RESEARCH  TRIANGLE  PARK  NC 
27709-2211 

1  US  ARMY  SOLDIER  &  BIO  CHEM  CTR 

ATTN  AMSSB  RT  P  W  LOEROP 
EDGEWOOD  CHEM  &  BIO  CTR 
BLDG  E  5554 
APG  MD  21010-5424 

1  COMMANDER 

USAISEC 

ATTN  AMSEL  TD  BLAU 
BUILDING  61801 
FT  HUACHUCA  AZ  85613-5300 

1  AFRL/SNAA 

ATTN  M  JARRATT 

2241  AVIONICS  CIRCLE  AREA  B 

BLDG  620 

WRIGHT  PATTERSON  AFB  OH 
45433-7321 

1  CMTCO 

ATTN  MAJ  A  SUZUKI 
1030  S  HIGHWAY  A1A 
PATRICK  AFB  FL  23925-3002 
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1  US  GOVERNMENT  PRINT  OFF 

DEPOSITORY  RECEIVING  SECTION 
ATTN  MAIL  STOP  ID  AD  I  TATE 
732  NORTH  CAPITOL  ST  NW 
WASHINGTON  DC  20402 

3  SITAC 

ATTN  H  STILES 
ATTN  K  WHITE 
ATTN  R  DOWNIE 

11981  LEE  IACKSON  MEMORIAL  HWY 
SUITE  500 

FAIRFAX  VA  22033-3309 

1  US  ARMY  RSRCH  LAB 

ATTN  AMSRD  ARL  WM  BF  G  HAAS 
APG  MD  21005-5067 

1  US  ARMY  RSRCH  LAB 

ATTN  AMSRD  ARL  WM  BF 
W  OBERLE 
BLDG  390 

APG  MD  21005-5067 


US  ARMY  RSRCH  LAB 
ATTN  AMSRD  ARL  RO  EV 
W  D  BACH 
POBOX  12211 

RESEARCH  TRIANGLE  PARK  NC  27709 

1 1  US  ARMY  RSRCH  LAB 

ATTN  AMSRD  ARL  Cl  OK  T 
TECHL  PUB  (2  COPIES) 

ATTN  AMSRD  ARL  Cl  OK  TL 
TECHL  LIB  (2  COPIES) 

ATTN  AMSRD  ARL  DIM  MILLER 
ATTN  AMSRD  ARL  SE 
I  PELLEGRINO 

ATTN  AMSRD  ARL  SE  S  I EICKE 
ATTN  AMSRD  ARL  SE  SE 
N  NASRABADI 
ATTN  AMSRD  ARL  SE  SE 
P  GILLESPIE 

ATTN  AMSRD  ARL  SE  SE 
S  S  YOUNG 
ATTN  IMNE  ALC  IMS 
MAIL  &  RECORDS  MGMT 
ADELPHI  MD  20783-1197 

TOTAL:  60  (59  HCs  and  1  Electronic) 


1  DIRECTOR 
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